Papers with generation speed
A Study of Non-autoregressive Model for Sequence Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Non-autoregressive (NAR) models generate all tokens in parallel, resulting in faster generation speed compared to autoregressive models. |
| Approach: | They propose to use knowledge distillation and source-target alignment to bridge the gap between NAR and autoregressive models in various tasks. |
| Outcome: | The proposed techniques can speed up NAR models in some tasks but not all . the proposed techniques reduce target token dependency while allowing for faster inference . |
WaveFM: A High-Fidelity and Efficient Vocoder Based on Flow Matching (2025.naacl-long)
Copied to clipboard
| Challenge: | Flow matching is a robust and stable approach to training diffusion models, but it can result in subpar audio quality. |
| Approach: | They propose a reparameterized flow matching model for mel-spectrogram conditioned speech synthesis that uses a mel prior instead of a standard Gaussian prior to minimize unnecessary transportation costs. |
| Outcome: | The proposed model improves sample quality and generation speed for speech vocoders while reducing transportation costs. |
Taylor Unswift: Secured Weight Release for Large Language Models via Taylor Expansion (2024.emnlp-main)
Copied to clipboard
Guanchu Wang, Yu-Neng Chuang, Ruixiang Tang, Shaochen Zhong, Jiayi Yuan, Hongye Jin, Zirui Liu, Vipin Chaudhary, Shuai Xu, James Caverlee, Xia Hu
| Challenge: | Existing mechanisms compromise ownership rights or raise data privacy concerns . existing mechanisms compromise security of released large language models . |
| Approach: | They propose a TaylorMLP to preserve the ownership of large language models by transforming the weights of LLMs into Taylor-series parameters instead of releasing original weights . |
| Outcome: | The proposed model preserves ownership of large language models and prevents their abuse by adjusting the generation speed and causing low-speed token generation. |
Decoding at the Speed of Thought: Harnessing Parallel Decoding of Lexical Units for LLMs (2024.lrec-main)
Copied to clipboard
Chenxi Sun, Hongzhi Zhang, Zijia Lin, Jingyuan Zhang, Fuzheng Zhang, Zhongyuan Wang, Bin Chen, Chengru Song, Di Zhang, Kun Gai, Deyi Xiong
| Challenge: | Large language models have demonstrated exceptional capability in natural language understanding and generation, but their generation speed is limited by the inherently sequential nature of their decoding process. |
| Approach: | They propose a method that accelerates decoding process without sacrificing quality . they propose lexical unit decoding, which can be integrated with other methods . |
| Outcome: | The proposed method significantly reduces decoding time while maintaining quality while maintaining output quality. |
DrDiff: Dynamic Routing Diffusion with Hierarchical Attention for Breaking the Efficiency-Quality Trade-off (2025.emnlp-main)
Copied to clipboard
Jusheng Zhang, Yijia Fan, Kaitong Cai, Zimeng Huang, Xiaofei Sun, Jian Wang, Chengpei Tang, Keze Wang
| Challenge: | et al., 2019; Brown e.t al, 2023; Touvron e t al; 2024; OpenAI, 2024) Large Language Models (LLMs) have demonstrated remarkable capabilities in knowledge encoding and contextual understanding during their pretraining phase. |
| Approach: | They propose a dynamic expert scheduling mechanism that allocates computational resources based on text complexity and a hierarchical sparse attention mechanism that adjusts attention patterns according to a variety of input lengths. |
| Outcome: | The proposed framework overpowers existing methods on long-text generation benchmarks. |
Revisiting Over-Smoothness in Text to Speech (2022.acl-long)
Copied to clipboard
| Challenge: | Non-autoregressive text to speech models ignore correlation in time and frequency domains, causing blurry results. |
| Approach: | They revisit the problem of over-smoothness in non-autoregressive text to speech models . they use methods that reduce complexity of data distributions and improve modeling methods . |
| Outcome: | The proposed models achieve better voice quality and faster inference speed than autoregressive models. |
Few-shot Temporal Pruning Accelerates Diffusion Models for Text Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing acceleration methods for text generation ignore the importance of the distribution of sampling steps, resulting in slow sampling rates. |
| Approach: | They propose a technique to accelerate diffusion models for text generation without additional training by using a Bayesian optimization approach. |
| Outcome: | The proposed technique achieves 400x acceleration even with minimal sampling steps after down to less than 1 minute of optimization yielding a competitive performance even with minimum sampling steps. |
Accelerating Multilingual Language Model for Excessively Tokenized Languages (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in large language models (LLMs) have shown a significant degree of multilingual proficiency on a variety of tasks in multiple languages. |
| Approach: | They propose a framework to fine-tune a language model head and fine-track it while preserving its performance. |
| Outcome: | The proposed framework increases the generation speed by 1.7 while maintaining the performance of pre-trained multilingual models on target monolingual tasks. |
Spec-VLA: Speculative Decoding for Vision-Language-Action Models with Relaxed Acceptance (2025.emnlp-main)
Copied to clipboard
| Challenge: | Visual Language Models (VLMs) have significant parameter size and autoregressive (AR) decoding nature impose considerable computational demands on VLA models. |
| Approach: | They propose a framework to relax acceptance utilizing the relative distances represented by the action tokens of the VLA model. |
| Outcome: | Empirical results show that the proposed framework improves the speed of the prediction task by 44%. |